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Abstract 

A correspondence is established between the elements of logic rea- 
soning systems (knowledge bases, rules, inference and queries) and the 
hardware and dynamical operations of neural networks. The correspon- 
dence is framed as a general translation dictionary which, hopefully, will 
allow to go back and forth between symbolic and network formulations, a 
desirable step in learning-oriented systems and multicomputer networks. 

In the framework of Horn clause logics it is found that atomic propo- 
sitions with n arguments correspond to nodes with n-th order synapses, 
rules to synaptic intensity constraints, forward chaining to synaptic dy- 
namics and queries either to simple node activation or to a query tensor 
dynamics. 



1 Introduction 

Neural networks and logical reasoning systems, because they both rely on Turing 
models of computation, are equivalent in the sense that whatever is computable 
in one framework must also be computable in the other. How to establish the 
equivalence in each particular case is however a non-trivial and interesting issue, 
because problems that are addressed in a simple manner in one approach turn 
out to be intractable in the other and vice- versa. For example, combinatorial op- 
timization problems are efficiently handled by neural networks, whereas logical 
systems succumb to combinatorial explosion. Conversely, for highly structured 
problems where appropriate heuristics are known, it is simpler and faster to use 
a logical reasoning system, the corresponding neural network system taking a 
long time to tune up because rule-based information is in general not so easy to 
build in the network architecture. 
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This situation led a number of authors [Q Q to propose the use of hybrid 
systems to solve complex problems, with a logic-like program for some parts 
of the process and a network for the rest. On the other hand, some authors 
have dealt with the identification of the network structures that might repre- 
sent the logical structures of a symbolic system^ {§ [§ f§ @] @] [[[3) Qand 
conversely with the nature of the rule-based information that may be extracted 
from networks. 

Extraction of rules from networks is an issue of practical importance in the 
construction of expert systems from example-trained networks Q fl^| . 
On the other hand when some prior rule-based information is known about a 
problem but nevertheless a network implementation seems appropriate, it would 
be useful to have some simple rules to implement the symbolic information on 
the architecture of the network. Neural networks take advantage of parallel 
VLSI hardware implementations, which largely improve the processing speed 
and it is not so clear how to take advantage of similar implementations for sym- 
bolic processing. Therefore even for problems that are naturally stated in logic 
terms, it might be useful to have a translation dictionary for hardware imple- 
mentation purposes. The establishment of a concise way to go back and forth 
between symbolic and network formulations is also welcome in learning-oriented 
systems and in the design of multicomputer networks. Different architectures 
make different types of learning easier or harder to design and in multicomputer 
networks it is essential for algorithms and architectures to fit together as well as 
possible. Finally and independently of the practical issues, the establishing of 
a compact translation dictionary between the two paradigms might be a useful 
step in the development of a unified language for cognitive processes. 

Most past attempts at establishing a logic-networks dictionary concern ques- 
tions of the architecture required to represent some types of logical operations or 
the refining of the numerical part of the knowledge base. In some cases, an exten- 
sion of the usual connectionist framework is required and a full correspondence 
is in general not established involving all the aspects of the systems (knowledge 
base, rules, inference, recursion and handling of queries). Here we describe an 
attempt to establish a correspondence involving all the basic elements that are 
present in logical and neural network systems. Recall that in a neural net- 
work one has a (distributed) memory on the connection strengths (synapses), 
a learning dynamics on the synapses and an activation dynamics of the node 
intensities. In a logical reasoning system there is some set of ground facts about 
the objects in the domain, a set of rules which are potential knowledge con- 
cerning relations between the objects and an inference mechanism (backward 
or forward chaining) allowing for the extraction of further information and the 
answering of queries. 

It has been argued that trying to isolate, in a network, the structures that 
correspond to specific logical statements or operations is a waste of time because 
everything in a network (memory, rules and inference) is distributed everywhere 
and forever inseparable. This may well be true for some architectures and some 
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classes of concepts. However even the identification of the modular structures 
that correspond to the logic elements is already a useful step. For example one 
of our conclusions here is that a natural network representation of an atomic 
proposition is a node with n th order synapses. Because of the universal approx- 
imation properties of neural networks, that same proposition might as well be 
represented with first order synapses, the proposition corresponding then to a 
submodule of several neurons. However the identification of the kind of minimal 
submodule that corresponds to a specific logical element is already a useful step. 

2 The network translation of a logic system 

For definiteness the scope of the logical systems, that is considered is a subset 
of the Prolog logic programming language. Namely the logic system is specified 
by a set of Horn clauses which are constructed from terms which are either 
constants or variables or structures. A constant is sensed to represent some 
concrete object in the domain of the problem. It is represented in the logical 
system by an indecomposable elementary symbol (an atom). Structures are 
restricted to be atomic propositions of the general form 



The functor is an atom and the parameter list is any list of atoms, variables or 
other atomic propositions. Finally a variable is an entity that can at any time 
be bound to any atom (constant or functor). Small letters will be used for the 
atoms and capitals for the variables. 

The first step is to find a network representation of the basic entities of 
the logical system. Each atom will correspond to a node. Therefore not only 
constants but also functors will be assigned a node in the network. 

The network nodes have output zero or one and threshold one. The nodes 
are connected in a network, with the inputs equipped with connection strengths 
(synapses) Wf^ lr .. yirl of order n|l5) [|l7) if n is the number of parameters in 
the parameter list of the functor. In particular a simple constant has no input. 
The node activation is 



where the polarization yf is zero unless there is a query involving this node. 
Fig. 1 displays the neural network representation of a constant node and Fig. 2 
that of a functor node. 

Whenever in the knowledge base, the relation 



functor (parameter list) 




(1) 



(2) 
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is stated as a true fact, the corresponding synaptic strength is 



(3) 




Otherwise, if the relation / does not hold for these particular set of pa- 
rameters the connection strength vanishes. This is the straightforward way to 
represent in the network the ground facts of the knowledge base, that is, the 
relations that are explicitly expressed among the atoms. On the other hand 
relations and clauses that involve variables correspond to constraints on the 
remaining connection strengths. This is better explained through an example. 
Let a knowledge base be 

f(b,c). 
m(a, c). 

m(d,a). . . 

p(X,Y) <= m(X,Y). K ' 

P (X,Y) <= f(X,Y). 

g(X,Z) 4= p(X,Y)Ap(Y,Z). 

The ground facts (the first three headless clauses) imply 

Wf bc = 1; Wmac = 1; Wmda = 1 (5) 

all other Wjxy and W m xY being set to zero. Let the connection strengths 
take only the values zero and one. Then the last three clauses in (0) translate 
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Figure 2: Network representation of a functor 



into the following constraints on the remaining W p xy an d W 9 xy connection 
strengths 

WpXY > W m XY 

W P XY > WfXY (6) 
Wgxz > WpxyWpYz 

These constraints may be realized in the network by a parallel dynamical law. 
Define the potential function 

V i = \ E * ( W ^y - W P xy)+1 * ( W f*Y - W pXy) + 1 X ( w pXyW pYZ - W gXZ ) 

XY XY XYZ 

(7) 

where x( x ) — x 2 9{x). Now the strengths (|5|) fixed by the ground statements are 
considered as frozen and the others evolve in parallel according to a gradient 
descent dynamics 

dW p xY dV 1 dW g xY dV 1 

dt dWpxY ' dt OWgXY 

This dynamics drives the connection strengths to values zero or one satisfying 
the constraints (|^). Fig. 3 shows the dynamical evolution of the connection 
strenghts for the potential (0). 

The method used in this example holds in general and for each rule of the 
form 

g(X 1 ,... 1 X n )^l[Af l (Y 1 ,...,Yp t ) (9) 



5 




Figure 3: A connection strengths evolution example 



a term 

\ E X\\{W h Y 1 ...Y p% 'W g x 1 ...X r \ (10) 
{X\{Y\ \ i ) 

is added to the potential V\ that drives the connection strengths, the sum being 
over all the variable terms. 

We have therefore established a correspondence where atomic propositions 
with n parameters correspond to n th order nodes and the rules are implemented 
by a parallel dynamics of the synapses. Because the synapse dynamics is equiv- 
alent to a forward chaining step, the network now contains on its connection 
strengths all the available information about the problem. Therefore informa- 
tion may be retrieved by simply looking at the values of the connection strengths. 
Alternatively an explicit scheme to answer queries may be established in the fol- 
lowing way. 

We may consider two different types of queries, simple queries without vari- 
ables and queries with variables. In the first case, for example 

l4=9{a,b). (11) 

the question is to check whether the relation g holds for the atoms a and b. In 
this case it suffices to excite the polarizations y a and y\, of the nodes a and b and 
see whether the node g lights up. The same happens for more complex queries 
like, for example 

?*=g(h(a,b),c)Af(c,d). (12) 



G 



where now the polarizations to be excited are y a , yb, y c and yd and the answer 
to the query is positive if nodes g and / light up simultaneously. 
For queries involving variables, for example 



? 4= m(d,X) Ap(X, c). 



(13) 



the question is to find the instantiations of X, if any, that make this a true 
statement. The product of the connection strengths W m dxW p xc — ax defines 
a vector ax in atom space and the non-zero entries of the vector are the in- 
stantiations that positively satisfy the query. A similar scheme holds for queries 
involving several variables. For example, for 



? ^g(h(X,c),a)Af(c,Y). 



(14) 



ones defines a tensor (3xy — WghaWuxcWfcY > the non-zeros entries of which are 
the positive answers to the query. With all the connection strengths established 
in the network by the (forward chaining- like) dynamics of Eq.(||) it might be 
an easy matter (depending on the type of implementation used) to look up the 
non-zero entries of ax or (3xy ■ Alternatively one may regard the network 
connection strengths as potential knowledge and generate dynamically the non- 
zero entries of the tensors 



dax 
dt 



dV 2 
da x 



XY 



dt 



dV 3 



XY 



with 



V 2 



J2x {WmdxWpxc - a x ) 2 



= Exy (W gha W h xcW fcY - PxyY 



(15) 



(16) 



Regarding the network structure and connection strengths as potential, as op- 
posite to explicit knowledge, the dynamics of the query tensors defined above 
plays a role similar to backward chaining in logical programming. 

As in (|j)-([lOj) the generalization, of this dynamics in the space of query 
tensors, to general queries is straightforward. 

The whole construction is summarized in Table 1. 



Logical System 


Network 


Constant 


Single node 


Atomic proposition (n arguments) 


Node with n th -ovdeT synapses 


Rules 


Synaptic intensity constrains 


Rule application (forward chaining) 


Synaptic dynamics 


Queries 


Node activation / Query tensor dynamics 



Table 1 : Logical system - Neural network translation table 

As a simple example we discuss problem 148 of Rcf. J2^] (see Appendix). 
Let us consider the following ground facts: 
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phone (coleman, '100001 ') 
phonefgordon, '100002'). 
phone(wagner, '100003'). 
phone(smith,'100004'). 
visits ( coleman, wagner) . 
at(wagner,gordon). 



The clauses in this example define the following potential 



VI SI 



tsXYWatYZ 



W atX z) + -Z ^ X(WatUvW phoneV N ~ W, 




XYZ 



UVN 



(17) 



The evolution of the connection strengths, according to the gradient descen- 
dent dynamics, leads to the neural network in Fig. 4, where only the non-zero 
connections strengths are shown. The continuous lines represent the connec- 
tion strenghts of the ground facts and the dashed ones the connection straights 
obtained from the clauses used to construct the potential V4. 

3 Conclusion and remarks 

The construction in the previous section emphasized the establishment of a 
correspondence between the pure logic elements of logic programming and both 
the hardware and dynamical operations of neural networks. Logic programming 
systems however, in addition to its pure logic elements, involve some non-logical 
features like the handling of arithmetics and procedural prescriptions for com- 
puting efficiency. These features need not concern us here because numerical 
capabilities are native to networks and procedural rules are implementation de- 
pendent. 

A more serious limitation concerns the restriction to Horn clauses. Horn 
clause logic is quite expressive and of wide computing power. However it cannot 
handle negation in a satisfactory way, especially for non-exhaustive databases. 
In some cases it is acceptable to reduce, by renaming Q, the set of non-Horn 
to a Horn clause set. However, depending on the nature of the database, it may 
be more convenient to retain a certain degree of non-Horness and to establish 
an inference procedure through case analysis jl^] [Mj. At a formal level, 
and to some extent, our formalism might be extended to non-Horn clauses. For 
example, if a non-Horn clause is 



h(X, Z) V g(X, Z) <= p(X, Y) A p(Y, Z). 
the relation for the connection strengths that extends (||) is 



(18) 



W h xz + W g xz > W p xyWpyz 



(19) 
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However it is not clear, at this time, how to implement dynamically, in an 
elegant manner, the case analysis mechanism. 

In this paper we have been concerned with logical reasoning systems in its 
sharp logics version. A natural concern is the extension to reasoning with un- 
certainty. Given that the handling of real numbers is quite natural in networks, 
an extension where all quantities (node activations and connection strengths), 
instead of being or 1, take a continuous range of values, does not seem to 
raise any special difficulties. Notice however that the straightforward extension 
of our network operations, as defined in Sect. 2, lead to represent the fuzzy 
intersection by a product rather than by the min operation. 



4 Appendix 

Problem 184 in Rcf. p2[| , originally suggested in "The Thinking Computer' 
by Bertrand Raphael, considers the following functors: 



number(X,N) means that person X can be reached by calling phone number N. 

visits (X, Y) means that person X is visiting person Y. 

at(X, Y) means that person X is at the residence of person Y. 

phone(X,N) means that person X has phone number N. 

and the following clauses: 

at(X,Z) <= visits(X,Y) A at(Y,Z) 

number (V ,N) <S= at(U,V) A phone(V,N) 

The purpose is to reach a person, having a phone number data base and 
knowing who visits whom and where the visited person is. 
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